Building Footprint Extraction Research Articles

Transformer-based semantic segmentation for large-scale building footprint extraction from very-high resolution satellite images

Extracting building footprints from extensive very-high spatial resolution (VHSR) remote sensing data is crucial for diverse applications, including surveying, urban studies, population estimation, identification of informal settlements, and disaster management. Although convolutional neural networks (CNNs) are commonly utilized for this purpose, their effectiveness is constrained by limitations in capturing long-range relationships and contextual details due to the localized nature of convolution operations. This study introduces the masked-attention mask transformer (Mask2Former), based on the Swin Transformer, for building footprint extraction from large-scale satellite imagery. To enhance the capture of large-scale semantic information and extract multiscale features, a hierarchical vision transformer with shifted windows (Swin Transformer) serves as the backbone network. An extensive analysis compares the efficiency and generalizability of Mask2Former with four CNN models (PSPNet, DeepLabV3+, UpperNet-ConvNext, and SegNeXt) and two transformer-based models (UpperNet-Swin and SegFormer) featuring different complexities. Results reveal superior performance of transformer-based models over CNN-based counterparts, showcasing exceptional generalization across diverse testing areas with varying building structures, heights, and sizes. Specifically, Mask2Former with the Swin transformer backbone achieves a mean intersection over union between 88% and 93%, along with a mean F-score (mF-score) ranging from 91% to 96.35% across various urban landscapes.

Unmanned Aerial Systems for Building Footprint Extraction in Urban Area

Unmanned Aerial Systems for Building Footprint Extraction in Urban Area

HD-Net: High-resolution decoupled network for building footprint extraction via deeply supervised body and boundary decomposition

The extraction of building footprints, as a highly challenging task in remote sensing (RS) image-based geospatial object detection and recognition, holds significant importance. Due to the strong coupling in RS images between the body and boundary of buildings, the ability of most currently advanced deep learning models in building footprint extraction remains limited, inevitably meeting the extraction performance bottleneck. To this end, we propose a novel High-resolution Decoupled Network, HD-Net for short, for precious building footprint extraction in RS. HD-Net follows the well-known high-resolution network (HRNet) architecture, which can to a great extent alleviate the coupling issues between body and boundary using its multi-scale information interaction in parallel. More specifically, Our HD-Net innovatively designs the multiple stacked multi-scale feature fusion (MFF) modules, where the MFF module is performed by combining the deep supervision technique and a feature decoupling–recoupling (FDR) module. The FDR module adeptly untangles coupled features into two distinct elements: body and boundary, yielding feature maps enriched with semantic information. This configuration facilitates a step-wise refinement of building extraction and boundary predictions, ensuring the overall continuity of buildings and the precision of their boundaries. Experiments conducted on the three widely-used building datasets, i.e., Massachusetts, WHU, and Inria, demonstrate that HD-Net achieves the most competitive results with minimal parameter count. In detail, HD-Net outperforms contour-guided and local structure-aware network (CGSANet) with intersection over union (IoU) improvements of 0.40%, 0.95%, and 0.73% on the mentioned datasets, while CGSANet is the state-of-the-art algorithm using a hybrid loss function and deep supervision strategy. Furthermore, the code of the HD-Net will be made available freely at https://github.com/danfenghong/ISPRS_HD-Net.

MERLIN-Seg: Self-supervised despeckling for label-efficient semantic segmentation

Remote sensing satellites acquire a continuous stream of data on a daily basis. As most of those data are unlabeled, the development of algorithms requiring weak supervision is of paramount importance. In this paper, we show that the need for annotation for Synthetic Aperture Radar data can be reduced by coupling a despeckling task (self-supervised) and a segmentation task (supervised). The proposed self-supervised learning framework, called MERLIN-Seg, has been trained for building footprint extraction and achieves favorable performances even with 1% of annotated data. We show that conditioning the network on despeckling without labels is beneficial for supervised segmentation. Our experiments demonstrate that the joint training of the two tasks achieves better performances than a vanilla segmentation network in terms of IoU, F1 score, and accuracy on both simulated and real SAR images.

Extracting Building Footprints in SAR Images via Distilling Boundary Information From Optical Images

Extracting Building Footprints in SAR Images via Distilling Boundary Information From Optical Images

Terrain-Informed Self-Supervised Learning: Enhancing Building Footprint Extraction From LiDAR Data With Limited Annotations

Terrain-Informed Self-Supervised Learning: Enhancing Building Footprint Extraction From LiDAR Data With Limited Annotations

Automatic building footprint extraction from photogrammetric and LiDAR point clouds using a novel improved-Octree approach

Extracting building footprints from optical data is a time-consuming process. Automatic extraction of building footprints from point clouds is a challenging problem in terms of geometric irregularities, noisy points, points density, and accuracy. The aim of this paper is to automatically extract and regularize building footprints using point clouds with a new approach called Improved Octree (I-Octree) by modifying the Octree method. The method consists of the separation of ground and above ground objects from the point cloud by Simple Morphological Filter (SMRF), the removing noisy points from point cloud with Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm, the automatic extraction of building footprints by I-Octree, and the regularization of the building footprints with Automatic Building Outline Regularization (ABORE) method. The proposed approach was implemented on photogrammetric and Light Detection and Ranging (LiDAR) in four test areas. Ground truth maps were utilized as reference data for accuracy analysis by using pixel-based accuracy method. The accuracy results were above 90 % for the photogrammetric point clouds and above 97 % for the LiDAR point cloud. It was proven that the proposed approach can extract and regularize the selected buildings with high accuracy compared the studies in literature. In conclusion, it was demonstrated that the proposed approach enables the automatic extraction and regularization of building footprints from point clouds. Consequently, the map production process with point cloud data is facilitated to be both more efficient and rapid, and the results confirm the high efficacy of the proposed approach.

Classification Analysis of Industrial Buildings Using Orthophoto and NDSM Data with Deep Learning Approach (Case Study: Kali Rungkut Village, Surabaya)

Information of industrial estates is important because it may increase the economy level, industrial goods production, and export activities. In addition, information of industrial areas needs to be identified so that the area does not interfere with agricultural productivity, natural resources, and cultural heritage. To obtain this information, extraction and classification of building footprints using orthophoto data with a deep learning approach is carried out. However, this has the challenge that the condition of the building is highly diverse in both shape and size, so it requires additional data such as height data form (NDSM) to facilitate its identification. The Mask Region-based Convolutional Neural Network (Mask R-CNN) method used for extraction produces 88.49% precision accuracy; 66.82% completeness (recall); and 76.15% F1-score. The classification model performed with the Deep Neural Network (DNN) method, produced excellent accuracy with average values of precision, recall, and F1-score of 0.94; 0.90; and 0.92, respectively.

Multiscale probability map guided index pooling with attention-based learning for road and building segmentation

Efficient road and building footprint extraction from satellite images are predominant in many remote sensing applications. However, precise segmentation map extraction is quite challenging due to the diverse building structures camouflaged by trees, similar spectral responses between the roads and buildings, and occlusions by heterogeneous traffic over the roads. Existing convolutional neural network (CNN)-based methods focus on either enriched spatial semantics learning for the building extraction or the fine-grained road topology extraction. The profound semantic information loss due to the traditional pooling mechanisms in CNN generates fragmented and disconnected road maps and poorly segmented boundaries for the densely spaced small buildings in complex surroundings. In this paper, we propose a novel attention-aware segmentation framework, Multi-Scale Supervised Dilated Multiple-Path Attention Network (MSSDMPA-Net), equipped with two new modules Dynamic Attention Map Guided Index Pooling (DAMIP) and Dynamic Attention Map Guided Spatial and Channel Attention (DAMSCA) to precisely extract the building footprints and road maps from remotely sensed images. DAMIP mines the salient features by employing a novel index pooling mechanism to retain important geometric information. On the other hand, DAMSCA simultaneously extracts the multi-scale spatial and spectral features. Besides, using dilated convolution and multi-scale deep supervision in optimizing MSSDMPA-Net helps achieve stellar performance. Experimental results over the seven benchmark building and road extraction datasets namely, Porto, Shanghai, Massachusetts Road, Massachusetts Building, Synthinel-1, WHU Satellite I and WHU aerial Imagery dataset, ensures MSSDMPA-Net as the state-of-the-art (SOTA) method for building and road extraction as our method beats the next best alternatives by 5.94%, 2.55%, 3.97%, 11.64%, 6.86%, 6.92%, 2.57% IOU and 3.98%, 1.90%, 2.43%, 7.17%, 3.98%, 4.99%, 1.37% F1 score, respectively, over the mentioned datasets. The code has been released in: https://github.com/shirshabose/MSSDMPA-Net.

SuperpixelGraph: Semi-automatic generation of building footprint through semantic-sensitive superpixel and neural graph networks

Most urban applications necessitate building footprints in the form of concise vector graphics with sharp boundaries rather than pixel-wise raster images. This need contrasts with the majority of existing methods, which typically generate over-smoothed footprint polygons. Editing these automatically produced polygons can be inefficient, if not more time-consuming than manual digitization. This paper introduces a semi-automatic approach for building footprint extraction through semantically-sensitive superpixels and neural graph networks. We first learn to generate superpixels that are not only boundary-preserving but also semantically-sensitive. The superpixels respond exclusively to building boundaries rather than other natural objects. These intermediate superpixel representations can be naturally considered as nodes within a graph. Consequently, graph neural networks are employed to model the global interactions among all superpixels and enhance the representativeness of node features for building segmentation, which also enables efficient editing of segmentation results. Classical approaches are utilized to extract and regularize boundaries for the vectorized building footprints. We efficiently accomplish accurate segmentation outcomes by utilizing minimal clicks and straightforward strokes, eliminating the necessity for editing polygon vertices. A significant improvement of 8% in AP50 was observed in vector graphics evaluation, surpassing established techniques. Additionally, we have devised an optimized and sophisticated pipeline for interactive editing, poised to further augment the overall quality of the results. The code for training the superpixel and graph networks will be made publicly available upon publication.11https://vrlab.org.cn/~hanhu/projects/spgraph/.

DFANet: Denoising Frequency Attention Network for Building Footprint Extraction in Very-High-Resolution Remote Sensing Images

With the rapid development of very-high-resolution (VHR) remote-sensing technology, automatic identification and extraction of building footprints are significant for tracking urban development and evolution. Nevertheless, while VHR can more accurately characterize the details of buildings, it also inevitably enhances the background interference and noise information, which degrades the fine-grained detection of building footprints. In order to tackle the above issues, the attention mechanism is intensively exploited to provide a feasible solution. The attention mechanism is a computational intelligence technique inspired by the biological vision system capable of rapidly and automatically catching critical information. On the basis of the a priori frequency difference of different ground objects, we propose the denoising frequency attention network (DFANet) for building footprint extraction in VHR images. Specifically, we design the denoising frequency attention module and pyramid pooling module, which are embedded into the encoder–decoder network architecture. The denoising frequency attention module enables efficient filtering of high-frequency noises in the feature maps and enhancement of the frequency information related to buildings. In addition, the pyramid pooling module is leveraged to strengthen the adaptability and robustness of buildings at different scales. Experimental results of two commonly used real datasets demonstrate the effectiveness and superiority of the proposed method; the visualization and analysis also prove the critical role of the proposal.

A Building Footprint Extraction from UAV Imagery Using Deep Learning.

A Building Footprint Extraction from UAV Imagery Using Deep Learning.

Integrating object-based and pixel-based segmentation for building footprint extraction from satellite images

Accurately delineating building footprints from optical satellite imagery presents a formidable challenge, particularly in urban settings characterized by intricate and diverse structures. Consequently, enhancing the utility of these images for geospatial data updates demands meticulous refinement. Machine learning algorithms have made notable contributions in this context, yet the pursuit of precision remains an ongoing challenge. This paper aims to enhance the accuracy of building footprint extraction through the integration of object-based and pixel-based segmentation techniques. Additionally, it evaluates the performance of machine learning methodologies, specifically LightGBM, XGBoost, and Neural Network (NN) approaches. The model's evaluation employed low spectral resolution optical images, widely accessible and cost-effective for acquisition. The study's outcomes demonstrate a substantial enhancement in extraction accuracy compared to extant literature. The proposed methodology attains an overall accuracy of 99.39%, an F1 measurement of 0.9935, and a Cohen Kappa index of 0.9870. Thus, the proposed approach signifies a noteworthy advancement over existing techniques for building footprint extraction from high-resolution optical imagery.

F2BFE: development of feature-based building footprint extraction by remote sensing data and GEE

ABSTRACT Monitoring the spatiotemporal dynamics of building footprints (BF) is necessary for understanding urbanization growth. It is a difficult task to extract residential sites, mainly BF, because of the complexity of their makeup and spectral variety. Additionally, conventional methods for building mapping typically rely on abundant training data and expertise from human operates. This study presents a new unsupervised Feature-Based Building Footprint Extraction (F2BFE) strategy using Sentinel-1&2 satellite images and the SRTM Digital Elevation Model (DEM). The newly developed radar index (NRI) from Sentinel-1 images was utilized to extract the Primary Building Footprints (PBF) through histogram analysis and thresholding techniques, based on the mean of annual Sentinel-1 VV and VH Backscatter channels in the Ascending orbit. In this research, the integration of the Otsu and Unimodal thresholding technique was developed as an optimal thresholding method for feature extraction. Furthermore, Sentinel-2 images were applied to extract spectral indices related to vegetation (NDVI, GNDVI, RDVI indices), water (NDWI index), and residential/built-up (NDBI, BuEI). The qualitative and quantitative validation results indicate that the NRI-based BF map achieved higher Overall Accuracy (OA) values of 98.14%, 90%, and 91% in Region of Interest-1 (ROI-1), ROI-2, and ROI-3, respectively. Additionally, the Kappa Coefficients (KC) for these regions were 0.96, 0.97, and 0.85, respectively. The NRI index provides an excellent OA result when vegetation, water, and slope features are carefully eliminated. Finally, it can be inferred that the simultaneous use of the sentinel-1&2 and slope data in feature space leads to increased BF accuracy.

Building Footprint Extraction from LiDAR Data Using SVM Classification

Automatic detection of buildings from very high resolution (VHR) satellite images is a current research hotspot in remote sensing. In this work presents a novel approach to detect the buildings by atomization of the training area collecting stage for supervised classification. The classification method for the analysis of satellite image based on Normalized Difference Vegetation Index (NDVI). The method employs the multi-spectral remote sensing data technique to find spectral signature of different objects such as vegetation index, land cover classification, concrete structure, road structure, rural and urban areas, rocky areas and remaining areas presented in the image. In the experiments, the proposed SVM framework outperformed for the three (Linear SVM, Cubic SVM, Medium Gaussian SVM) methods tested. SVM is trained through supervised learning to classify each location in the image as “buildings” or “Non-buildings.” Finally, SVM algorithm to find the accuracy value, training time and receiver operating characteristic curves to evaluate detection performance, and compare the proposed algorithm with several existing methods.

Building footprint extraction from aerial photogrammetric point cloud data using its geometric features

Unmanned aerial vehicles (UAVs) can quickly acquire high-resolution datasets. Generally, UAVs or drones have high-resolution optical cameras onboard to obtain aerial images. These images are processed to provide various output products, including point cloud, digital surface model (DSM), digital terrain model (DTM), and ortho-mosaiced image. This study uses point cloud data obtained from UAV data processing to extract buildings automatically. It utilizes the geometric features obtainable from the point cloud data in a defined neighbourhood to classify the point cloud data. Normalized DSM (nDSM) is also used as an input to identify above-ground features more accurately. Random Forest (RF) algorithm has been used to classify the point cloud data into different classes available in the dataset. After classification, buildings are separated from the point cloud data, and K-Means clustering is performed to segregate different building clusters. These clusters are rasterized, and morphological operations are applied to refine the building edges. Then the boundaries of the building clusters are identified to provide the vector data. Accuracy assessment of the automatically extracted shapes is done by comparing their area, perimeter, and centroid location to the reference building polygons generated through the total station survey. The methodology is tested over the dataset acquired through UAV. An open-source GUI (graphical user interface) based tool has been developed in Python to extract the vectorized building shapes from photogrammetric point cloud data and compute areas automatically. It will reduce manual interventions significantly and benefit many users, professionals and researchers.

AN END-TO-END DEEP LEARNING WORKFLOW FOR BUILDING SEGMENTATION, BOUNDARY REGULARIZATION AND VECTORIZATION OF BUILDING FOOTPRINTS

Abstract. Automatic building footprint extraction from remote sensing imagery is a widely used method, with deep learning techniques being particularly effective. However, deep learning approaches still require additional post-processing steps due to pixel-wise predictions, that contribute to occluded and geometrically incorrectly segmented buildings. To address this issue, we propose an end-to-end workflow that utilizes binary semantic segmentation, regularization, and vectorization. We implement and assess the performance of four convolutional neural network architectures including U-Net, U-NetFormer, FT-UnetFormer, and DCSwin on the MapAI Precision in Building Segmentation competition. To additionally improve the shape of the predicted buildings we apply regularization on the predictions to assess whether regularization further improves the geometrical shape and improve the prediction accuracy. We aim to produce accurate predictions with regularized boundaries that can prove useful in many cartographic and engineering applications. The regularization and vectorization workflow is further developed into a working QGIS-plugin that can be used to extend the functionality of QGIS. Our aim is to provide an end-to-end workflow for building segmentation, regularization and vectorization.

Performance evaluation of three deep learning models in building footprint extraction from aerial and satellite images

Performance evaluation of three deep learning models in building footprint extraction from aerial and satellite images

Transformer-Based Semantic Segmentation for Extraction of Building Footprints from Very-High-Resolution Images.

Semantic segmentation with deep learning networks has become an important approach to the extraction of objects from very high-resolution remote sensing images. Vision Transformer networks have shown significant improvements in performance compared to traditional convolutional neural networks (CNNs) in semantic segmentation. Vision Transformer networks have different architectures to CNNs. Image patches, linear embedding, and multi-head self-attention (MHSA) are several of the main hyperparameters. How we should configure them for the extraction of objects in VHR images and how they affect the accuracy of networks are topics that have not been sufficiently investigated. This article explores the role of vision Transformer networks in the extraction of building footprints from very-high-resolution (VHR) images. Transformer-based models with different hyperparameter values were designed and compared, and their impact on accuracy was analyzed. The results show that smaller image patches and higher-dimension embeddings result in better accuracy. In addition, the Transformer-based network is shown to be scalable and can be trained with general-scale graphics processing units (GPUs) with comparable model sizes and training times to convolutional neural networks while achieving higher accuracy. The study provides valuable insights into the potential of vision Transformer networks in object extraction using VHR images.

Joint semantic–geometric learning for polygonal building segmentation from high-resolution remote sensing images

As a fundamental task for geographical information updating, 3D city modeling, and other critical applications, the automatic extraction of building footprints from high-resolution remote sensing images has been substantially explored and received increasing attention over recent years. Among different types of building extraction methods, the polygonal segmentation methods produce vector building polygons that are in a more realistic format compared with those obtained from pixel-wise semantic labeling and contour-based methods. However, existing polygonal building segmentation methods usually require a perfect segmentation map and a complex post-processing procedure to guarantee the polygonization quality, or produce inaccurate vertex prediction results that suffer from wrong vertex sequence, self-intersections, fixed vertex quantity, etc. In our previous work, we have proposed a method for polygonal building segmentation from remote sensing images that addresses the above limitations of existing methods. In this paper, we propose PolyCity, which further extends and improves our previous work in terms of the application scenario, methodology design, and experimental results. Our proposed PolyCity contains the following three components: (1) a pixel-wise multi-task network for learning the semantic and geometric information via three tasks, i.e., building segmentation, boundary prediction, and edge orientation prediction; (2) a simple but effective vertex selection module (VSM), which effectively bridges the gap between pixel-wise and graph-based models via transforming the segmentation map into valid polygon vertices; (3) a graph-based vertex refinement network (VRN) for automatically adjusting the coordinates of VSM-generated valid polygon vertices, producing the final building polygons with more precise vertices. Results on three large-scale building extraction datasets demonstrate that our proposed PolyCity generates vector building footprints with more accurate vertices, edges, shapes, etc., achieving significant vertex score improvements while maintaining high segmentation and boundary scores compared with the current state-of-the-art. The code of PolyCity will be released at https://github.com/liweijia/polycity.